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Careful sequencing of instructional stimuli has always been 
considered a highly important variable in programed instruc- 
tion. As early as 1954, Skinner suggested the logical necessity 
of sequence: “The device [teaching machine] makes it possible 
to present carefully designed material in which one problem can 
depend upon the answer to the preceding and where, therefore, 
the most efficient progress to an eventually complex repertoire 
can be made [p. 9o].” In 1958 Skinner wrote, ”.. . technical 
terms, facts, laws, principles, and cases . . . must then be arranged 
in a plausible developmental order [p. 974].” Basic to Skinner's 
approach is the notion that through empirical feedback from stu- 
dents to the programmer, there can be developed an ideal, or 
optimal, learning sequence (Lumsdaine, 1964, p. 383). 

Present day programmers, while gradually becoming less re- 
strictive in terms of many of Skinner’s theoretical presupposi- 
tions (overt responses, individual pacing, constructed responses, 





Fred C. Niedermeyer is an elementary and secondary school teacher of mathe- 
matics, and is currently in the Product Research Program, School of Education, 
University of California, Los Angeles. 


VOL. 16, NO. 3, FALL 1968 301 


RELATIONSHIP 
BETWEEN 

TYPE OF 
LEARNING AND 
EFFECT OF 
SEQUENCE 


AV COMMUNICATION REVIEW : 302 


etc.), still retain the notion of sequence. Note Susan Markle’s 
1967 definition: ‘We define an instructional program as a re- 
producible sequence of instructional events designed to produce 
a measurable and consistent effect on the behavior of each and 
every acceptable student [p. 104].” 

Given the weighting assigned to sequence in programed in- 
struction, experimenters over the past several years have been 
unable to resist the temptation to test the sequence hypothesis 
by comparing logically-ordered frame sequences with sequences 
whose “plausible developmental order” has been destroyed 
through randomization. At least eight of these so-called “scram- 
ble studies” have been reported since 1961 (Krathwohl, Payne, 
& Gordon, 1967; Levin & Baker, 1965; Miller, 1965; Nieder- 
meyer, Brown, & Sulzen, in press; Roe, 1962; Roe, Case, & Roe, 
1962; Stolurow, 1964; Wodtke, Brown, Sands, & Fredericks, 
1967), and they constitute a rather perplexing state of affairs: a 
box score would indicate that only one of them, Roe (1962), has 
revealed a significant advantage for a logically ordered sequence 
on learner posttest performance. However, such a box score con- 
clusion would be misleading since certain methodological weak- 
nesses may be identified in some of these studies. 

The purpose of this paper is to review this issue of frame 
sequence which compares the relative effectiveness of logically 
ordered frames versus randomized frame sequences in the learn- 
ing of classroom subject matter using programed instruction. 
Before proceeding with an analysis of the research, perhaps it 
would be wise to examine one of the underlying theoretical con- 
cepts regarding frame sequence—the interrelation between the 
task types being taught and the degree to which frame sequence 
affects learning. 

Among the first to conclude that program content was probably 
a critical factor in determining whether or not sequence would 
affect learning were Levin and Baker (1965) when discussing 
the “no difference” results of their 1963 scramble study using 
a second grade geometry program. Since then, authors of sequence 
studies have generally selected programs from mathematics, 
where, supposedly, there are many instances in which certain 
things have to be learned before others—i.e., where sequence 
should be important relative to achievement. The model usually 
employed to describe such hierarchical learning is Gagné’s (1965) 
concept of a knowledge structure where principles to be learned 
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depend upon previously acquired principles and concepts, which 
in turn depend upon previously learned discriminations among 
stimulus-response associations. 

It is argued that a program developed according to such a 
rational analysis of the subject matter would exhibit a high de- 
gree of sequential dependence among its frames as learners 
work upward through the hierarchy of tasks. Randomizing this 
logical, hierarchical sequence in such a program should cause 
Ss to make significantly more program errors, and, it is implied, 
less learning will take place. On the other hand, if a program 
consists of a series of items in which the correct responses were 
independent of each other—such as a list of unrelated spelling 
words or historical facts—then the overall program error rate 
should remain constant no matter how the frames are sequenced 
since the items form rather a flat structure instead of a hierar- 
chical or pyramid-shaped structure. Thus Krathwohl, Payne and 
Gordon (1967) state “’... there may be a continuum of depen- 
dence on sequence [p. 126].” 

Upon the basis of the above discussion, error rate differences 
are taken as an index of the degree to which logical sequencing 
is relevant in a particular learning program. Whenever a 
scramble study is reported in which logical and random program 
error rates do not differ very much (Krathwohl et al., 1967; 
Levin & Baker, 1965; Roe, Case, & Roe, 1962), it is argued 
that no learning differences should be expected since frame 
order did not seem to affect the learners’ ability to respond cor- 
rectly and thus there may not have been a necessity for any 
particular frame sequence in the program. 

Disciples of linear programing were interested in 1962 when 
Roe, Case and Roe found no posttest differences after com- 
paring normal order and scrambled sequences of a 71-frame 
program on elementary probability theory for college freshmen. 
The authors concluded that ’... for short programs careful se- 
quencing may not be important.” Since such a conclusion was 
(and still is) quite upsetting to popular programing beliefs, 
critics quite naturally began to look for flaws in the study. 
Holland (1967) in his 1965 review, pointed out that “program 
error rate for the two [sequence] forms did not differ, suggesting 
that the items were not highly interdependent even in the 
‘sequenced’ case, either because of the nature of the concepts 
taught or because of overcueing [p. 69].” 
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Shortly after this initial study, Roe revised the program add- 
ing 22 frames, and ran the experiment again comparing not 
only linear and random frame sequences but several types of 
branching sequences as well (Roe, 1962). This time the linear 
(logical) version did produce significantly better outcomes than 
the random version relative to error rate, time to complete the 
program, and posttest scores. Holland, in reviewing this second 
study, apparently accepted it as evidence enough, for he con- 
cluded that “... the need for sequenced material is supported, 
at least if the frames are interdependent and not severely over- 
cued,” and he further suggested, “What is needed now is not 
more studies of scrambled versus ordered sequences, but con- 
structive and creative research which will indicate when, how 
much, and in what way items may be sequenced [p. 69].” 

While Holland apparently considered the issue closed, an in- 
spection of the data from the second Roe study (1962) reveals 
some problems. The “significant difference” Roe found for the 
posttest scores turns out to be only ten percentage points (59.4 
percent versus 69.7 percent) which, on the 14-item test he used, 
is a difference of only one and one-half items. The significant 
learning-time difference was only 13 minutes on the two 
hour program (118 minutes versus 131 minutes). While these 
differences are statistically different, one wonders whether the 
magnitude of these differences is large enough to consider the 
issue closed. Another interesting outcome of this study was that 
a forward branching version of the logical sequence program 
administered via a programed text format (the others were ad- 
ministered via cards in file boxes) produced a mean six percent- 
age points lower than the scrambled program. Nevertheless this 
is the only study the writer is aware of in which the outcomes 
have been consistent with programing theory. 

Adding a new dimension to the previously discussed relation- 
ship between sequence and the type of material being taught, 
Stolurow in 1964 found an interesting interaction between IQ 
and sequence when comparing a mixed (random) sequence and 
a consecutive (logical) sequence designed to teach fractions to 
educationally handicapped high school students (mean mental 
age of 12 years). Only those with higher ability were successful 
using the mixed sequence, while all ability levels were success- 
ful with the consecutive ordering. Stolurow concluded that “... 
the best sequence [consecutive] did for the poorest ability group 
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what the highest ability groups could do for themselves regard- 
less of sequence.” Consequently, subsequent scramble studies to 
date have included the hypothesis that sequence effects may 
depend upon the learner’s ability—high ability learners should 
be able to overcome the effects of poor sequencing better than 
low ability learners. (A logical corollary of this would be that the 
age of the learner may be relevant, yet it seems that most se- 
quence studies have been conducted with college Ss.) 

Since Holland’s 1965 review, at least four new studies on the 
issue have been reported, and all are in the “no difference” 
category. 

Krathwohl, Payne and Gordon (1967) examined the effects 
of scrambling upon the learning of three college statistics pro- 
grams which ‘varied in the judged logical interrelatedness of 
the material in each program from low to fairly high [p. 126, 
emphasis mine].” They found no treatment effects and no in- 
teractions with ability. However, several problems may have 
hampered the validity of this study. Error rates for both logical 
and random sequence conditions ran between 4 percent and 6 
percent, and controls scored almost as high as treatment Ss. 
Also the instructional conditions were uncontrolled in that sub- 
jects were requested to complete the programs at home. 

Miller (1965) reported a study in which a 98-frame program 

on topics in ratio and proportion was presented in both logical 
and random sequences to middle-track seventh-graders. While 
error rates differed substantially (5 percent on logical versus 
13 percent on random), instructional time differed very little 
(50 minutes on logical versus 56 minutes on random), and cri- 
terion test scores differed not at all (both means were about 38 
on a 50-item test). Also the pretest mean was 29, indicating only 
9 items on the 50-item test. It would seem that the Ss knew quite 
a bit of the material prior to instruction, thus making it easier for 
them to overcome the effects of random sequencing. 
Wodtke, Brown, Sands and Fredericks (1967) recently reported 
two well controlled experiments where computer terminals were 
used and a comparison of random and logical sequences of in- 
struction in two content areas were made. 

In the first experiment, a 74-frame program on number bases 
was administered to 80 education majors at Pennsylvania 
State University. This program was thought to contain a concep- 
tual hierarchy where sequence would be important. As was ex- 
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pected, a significant difference in program error rates did ap- 
pear, favoring the logical order, and the experimenters took this 
to ‘’. .. support their contention that the ordered version of the 
modern mathematics program . . . did contain an ordered concep- 
tual sequence . . . [i.e.] the conceptual hierarchy model suggested 
by Gagné which implies that the mastery of subordinate tasks 
in the hierarchy is essential to mastery of tasks at higher levels 
in the hierarchy [p. 64].” Contrary to expectations, however, 
there were no achievement differences or aptitude-sequences in- 
teractions with respect to posttest performance. 

The second experiment utilized a program teaching discrete 
facts relative to the anatomy of the ear, and no effect for se- 
quence was hypothesized. The material apparently was non- 
hierarchical or flat in structure, and there was no error-rate 
difference, as well as no posttest difference, between sequence 
groups in this second experiment. 

This study contained several features that were an improve- 
ment over previous scramble studies. The number base pro- 
gram was an effective instrument for producing criterion be- 
haviors (mean of 18 on a 22-item posttest), and it dealt with 
knowledge for which the Ss had little or no previous background 
(90 percent of the Ss scored zero on pretest). The learning 
situation was extremely well controlled through the use of a com- 
puter teletype terminal. The achievement test was reliable (r = 
.93), and examination of it and the program indicated content 
validity. How then does one account for the fact that no differ- 
ences in achievement were found? The authors offered the follow- 
ing explanation: 

... adult Ss are able to relate the relevant information as it is made 
available by the [scrambled] program to the previously poorly 
understood concepts which were presented out of sequence . . . this 
interpretation leans heavily on the cognitive processes of the Ss 


which allow them to integrate and organize information regardless 
of the sequence of presentation [p. 31]. 


The authors contended that their data raised “some embarrassing 
questions” for programed instruction theorists: 


... we would venture the tentative conclusion on the basis of accu- 
mulated evidence to data [here they are referring to previously 
mentioned studies], that instructional sequencing may be an overrated 
variable... manipulations of instructional stimulus variables may 
account for only a very small portion of the variance accounted for 
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by the learning strategies and information processing strategies em- 
ployed by the students. [p. 67] 

After considering some of the conceptual and methodological 
problems in several of the above studies, and after admitting 
an intuitive unwillingness to believe the trend of “no differences,” 
the author, along with two colleagues, undertook still another 
scramble study. This experiment (Niedermeyer, Brown, & Sul- 
zen, in press) was recently completed and is reported briefly here. 
From the outset, we attempted to secure a learning program on 
which there existed empirical data as to the high degree of 
interdependence among the concepts and principles being taught, 
ie., where sequence had been proven to be highly relevant. 
Consequently a modified version of the Number Series program 
from Gagné and Brown’s 1961 study on discovery learning, 
“Some Factors in the Programing of Conceptual Learning,” was 
employed for this experiment. The program consisted of 70 in- 
troductory frames (there were 89 originally, but some of the repe- 
titious practice frames were removed due to time constraints) 
followed by 40 guided-discovery frames. 

These latter frames required the learner to “actively produce 
... concepts... which he had just previously learned in the 
introductory program” in order to attain the problem-solving 
skill of “finding a formula for the sum of n terms in a number 
series” (Gagné & Brown, 1961, p. 319). For example, in the dis- 
covery part of the program the learner would be given a num- 
ber series, such as 1 2 4 8 16 32 64 etc., and then would be 
guided through a series of ten frames to produce the formula 
for the first n terms in in this series, which happens to be 

n 

2=Ty41.—1. 
Presumably it would seem that in order to perform this task the 
learner would at least have to understand the symbolized con- 
cepts (2, n, T), principles interrelating these concepts, and sub- 
script notation—all of which are taught in the 7o introductory 
frames. 

This particular learning program was selected because it was 
thought to be based on the knowledge hierarchy empirically 
validated for this same terminal task (finding formulas) by 
Gagné (1962). However, it later became apparent that the Num- 
ber Series program must have been developed prior to Gagné’s 
derivation of this knowledge hierarchy since little correspondence 
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could be found between the frames of the learning program and 
the tasks identified in the hierarchy. This, unfortunately, pre- 
vented us from randomizing the presentation order of task units 
within the program, and instead we had to scramble all of the 
frames. We also decided that, besides destroying the logical se- 
quence in Gagné’s program by randomizing the frame order, the 
program would also be administered in reverse frame order— 
the rationale being, “If this fails to impair learner achievement, 
then we have some real thinking to do about frame sequence.” 
Sixteen ninth-grade algebra students in each of the three se- 
quence groups plus a control served as the subjects. Inprogram 
measures (see Table 1) indicated that there were no differences 
between sequence groups in the amount of time taken to com- 
plete the frames. Program error rates, however, did differ signif- 


’ Measure 
Sequence Time (Minutes) Program Errors 
Condition M SD M SD Error Rate 
Logical 78.5 15.3 38.2 16.6 35% 
Scrambled 80.9 13.6 54.8 17.0 50% 
Reverse 81.5 6.5 52.1 13.8 47% 


icantly (p < .05) in favor of the logical sequence group, thus 
indicating a substantial sequence effect in the program. The 
rather high magnitude of the error rates, even for the logical 
Ss, suggests that the program was quite difficult for this partic- 
ular sample of ninth graders. Much lower error rates were ob- 
tained for this same program in Gagné and Brown's study 
(1961) and in a replication of their study by Meconi (1967), 
but this could perhaps be explained by an aptitude difference 
in the samples. The IQ range for the present study was 93 to 
137 (mean of 112) whereas in the Gagné and Brown study the 
range was 100 to 154, and in the replication study (Meconi, 
1967) the mean was 128. 

Of course it could also be argued that the required responses 
in many of the frames were undetermined or uncued. Holland 
(1967) discusses this problem, and if it were valid for the 
Number Series program, then the effectiveness of the program 
to teach the required behaviors could be questioned. However, 
inspection of the frames indicates that undercueing was not 
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the problem, and one boy in the logical group made only nine 
program errors during the experiment. 

Results of postprogram measures for each sequence condition 
and the noninstructed controls are given in Table 2. The Test 





Measure 
10-Item Test of 10-Pont Transfer Test of 

Sequence Introductory Concepts Problem Solving Skill 
Condition M SD M SD 
Logical 5.58 31 3.38 3.0 

(N = 15) 

Scrambled 4.18 2.9 1.8 3.0 

(N = 16) 

Reverse 3.4 3.5 2.6° 2.1 

(N = 16) 

Control 0.8 1.0 0.3 0.7 

(N = 16) 


* Significantly different from Controls, p < .o5 


of Introductory Concepts consisted of ten tasks taken directly 
from the program, but applied to a new number series. 


QR 
example: If Tn = 9, then 2 = ——? 


The Transfer Test of Problem Solving consisted of five new num- 
ber series for which the S was expected to state a general equation 
that would give the sum of the series at any term, n. Three of 
these number series had been used as test items by Gagné and 
Brown, and the other two were developed by the experimenters. 
Responses to each of the five series were scored on a two-point 
basis for a total of ten points. 

For both tests, analysis of variance indicated significant dif- 
ferences among the means (p < .o1). A posteriori comparisons, 
however, using the Scheffé method (Ferguson, 1966, p. 296) 
to hold the probability of a Type I error constant, reveal that 
only the logical sequence group performed significantly better 
than the control group on both tests. The scrambled sequence 
group performed significantly better than the controls on the 
Test of Introductory Concepts but not on the Transfer Test. The 
reverse sequence group performed just the opposite. None of 
the three sequence groups differed significantly from each 
other on either test. These results, while in an encouraging direc- 
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tion, certainly are not sufficient for closure on the issue of frame 
sequence. 

The possibility of IQ and sequence interaction effects on the 

Test of Introductory Concepts was examined by means of a fac- 
torial analysis of variance and no significant effects for either 
sequence or sequence X IQ interaction were found. 
Figure 1 presents means of each sequence group on the Intro- 
ductory Test after the Ss had been split at the median IQ of 
114. While it is obvious that little interaction effect exists, it 
is interesting to note that for the upper half of the Ss relative 
to IQ the program appears fairly effective. Again it was con- 
cluded that the problem with the program was that it was too 
difficult for this particular sample of Ss and not that the pro- 
gram was simply ineffective. 

An attitude measure administered by the classroom teacher 
the day following the study revealed an interesting difference 


Mean Number of Correct Responses on Test of Introductory Concepts 


by High and Low IQ Ss Under Logical, Scrambled, and Reverse 
Sequence Conditions* 


(2.3) 
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Sequence Condition 


*N’s Equaled: Hi-IQ/Logical = 7, Hi-IQ/Scrambled = 9, Hi-IQ/Reverse 
= 6, Lo-IQ/Logical = 6, Lo-IQ/Scrambled = 7, Lo-IQ/Reverse = 9 
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among the groups. Ss in the logical sequence group tended to 
consider the program “interesting,” whereas Ss in the scrambled 
and reverse treatment groups tended to feel “neutral’’ about 
the program (x? = 6.6, p < .05). No differences were found rela- 
tive to how difficult the Ss felt the materials and the tests were. 
That the logical Ss felt the program was more interesting than 
Ss in the other two groups would be expected because their error 
rate on the program was substantially less. The program was 
not as punishing for Ss under the logical sequence condition. It 
is also possible the logical group Ss professed higher affect toward 
the program because they were able to follow the simple-to-com- 
plex development from concepts and principles to problem-solv- 
ing skills, whereas Ss in the other two groups were never able 
to figure out what the program ‘was driving at’ even though 
they learned to perform some of the tasks taught in the 
program. 

It was concluded, “Though sequence effects would be expected 
to appear in a fairly long instructional program, the present 
study, as well as previous studies, lead the authors to believe 
that for short (less than two hours) programs, sequence may not 
be as crucial to cognitive outcomes as has previously been 
thought.” We also noted, however, that the significant differ- 
ence in learner affect “may be enough to justify careful at- 
tention to frame sequence in short instructional programs” (Nie- 
dermeyer et al., in press). 

Anderson, in his 1967 review, argued that ‘no difference” re- 
sults from scramble studies could be explained in terms of a 
spaced practice effect: “It seems possible that the disruptive 
effect of presenting frames out of order was counterbalanced 
by the facilitative effects of larger intervals between similar 
frames [p. 157].” This writer would agree with Anderson when 
the task to be learned is a rather simple, one-step principle or 
an instance of a particular concept. It can be noted from the 
data in Table 2 that the random sequence group performed 
higher than the reverse order group on the Test of Introductory 
Concepts where the learner had to respond correctly to instances 
of recognizing and using concepts. (Example: If the term-value 
from the above number series is 8, then what is the correspond- 
ing term number?) Any of these tasks could be presented in 
a single frame and thus random group Ss may have benefited 
from the spaced practice effect advocated by Anderson. 
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On the other hand, it will also be noted that the random se- 
quence group was not even statistically superior to control Ss 
on the Problem Solving Test where they had to derive a formula 
by following a series of guided-discovery frames during the 
latter part of the program. Randomization apparently destroyed 
any guiding effects present in these series of frames. Reverse 
order Ss, however, did perform statistically better than controls 
on the Problem Solving Test. The guided discovery sequence of 
frames, while presented backwards to this group, apparently 
was not completely destroyed and Ss were able to garner some 
problem-solving competency. One boy in the reverse order group 
remarked during instruction, ‘Whenever I begin to understand 
what they are doing, they switch to something new.” What we 
are saying here is that sequence seems to have been more im- 
portant in the latter, problem-solving part of the program than 
in the introductory part where each frame represented a simple, 
discrete task. 

In wondering how reverse order Ss could even attempt the 
problem solving (finding formulas) frames without having gone 
through the introductory frames which taught the supposedly 
prerequisite concepts, it could be argued that these Ss figured 
out for themselves the prerequisite behaviors by using knowl- 
edge of correct answers provided on the back of each frame. Many 
reverse order Ss were observed spending considerable time look- 
ing back at a frame once they had exposed the correct answer. 
In such a manner they may have been able to comprehend 
the meaning of the symbols used for writing the formulas. Some 
Ss, on the other hand, did not use knowledge of correct answers 
to look back at a frame and fill in the gaps. Instead, they 
would merely register a facial expression that signified displea- 
sure (they were wrong or unable to answer 50 percent of the 
time) and move immediately to the next frame. 

The experimenters found the above phenomenon quite inter- 
esting and a study could be planned in which, on missed frames, 
latencies after receiving knowledge of correct answers before 
going on to the succeeding frame are measured. It might be 
hypothesized that those Ss showing the greatest persistence 
would probably learn the most. If this hypothesis held up, it 
could be argued that problem-solving programs, where prereq- 
uisite skills must be searched out by the learner himself, might 
be a useful application of programed instruction relative to pro- 
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moting learning-how-to-learn behaviors. Logically-ordered, 
small-step, minimal-error-rate programs that are held in such 
high esteem at present may effectively promote desired cognitive 
outcomes, but, through overuse of spoonfeeding through se- 
quence, may not promote desirable study skills and searching 
behaviors. This, fortunately, is an empirical question and one 
that may be worthy of investigation in the future. Two girls in 
the study (Niedermeyer et al., in press) worked through the pro- 
gram in reverse order and made perfect scores on both posttests. 
One of these had an error rate of only 15 percent. How did they 
do it? Control group scores indicate little chance of prior experi- 
ence with the program content. If we could identify the informa- 
tion-processing strategies they must have employed, then there 
exists the possibility that we could actually teach these behaviors 
to others. 

While many of the studies on the issue of frame sequence 
possess certain methodological deficiencies, some of them—espe- 
cially the Wodtke et al. (1967) study—seem valid enough to 
agree with Evans (1965) that “’... it is possible that with very 
short programs, scrambling the order of the items may make 
little difference [p. 385].”” While Evans also states, “’.. . it seems 
highly unlikely that any successful, well-revised program of 
more than 100 frames in length, in highly structured topics 
such as mathematics or logic, could be successfully scrambled 
in its entirety and still do the job it was designed to do,” we 
would once again point out that the evidence accumulated to 
date implies that programs considerably longer than Evans sug- 
gests may be relatively unaffected by frame sequence. We can- 
not concur with programing theorists who seem to infer that care- 
ful regard for optimal sequencing is crucial in all cases. Indeed 
it has already been argued that careful frame sequencing which 
leads learners through subject matter with a minimum of frame 
errors may be detrimental to certain desirable outcomes related 
to learning how to learn. 

Since the writer seems to be discounting (though certainly not 
dismissing) the role of frame sequence in short programs, one 
might wonder just what it is he does consider crucial to learning 
outcomes. In attempting to answer this question, let me first at- 
tempt to clarify what seems to be a confusion between Gagné’s 
hierarchical learning structures, on one hand, and program frame 
sequence on the other. Several of Gagné’s experiments (Gagné, 
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1962; Gagné, Mayor, Garstens & Paradise, 1962; Gagné & 
Paradise, 1961; Gagné & staff, 1965), each containing a rather 
complex task hierarchy, have shown, indeed, that once a learner 
fails when tested on a particular task, he also fails all higher 
level tasks in the structure. It is easy to see how people might 
interpret this as synonymous with the sequence in which in- 
structional stimuli must be presented to learners. Yet this 
simply is not true. It has been shown in several of the previously 
reviewed studies that instruction relative to prerequisite tasks 
forming a learning structure can be presented in a random or 
even a reverse order without substantially affecting the amount 
of end-of-program learning that takes place. 

What Gagné is saying, quite accurately, is that some things 
must be learned before others—the concepts that comprise a 
principle must be acquired before the learner can show his 
understanding of the principle, etc.—yet this sequence of learn- 
ing is quite different from sequence of instructional frames or 
stimuli. Although Ss may miss frames requiring the under- 
standing of a principle when they do not have mastery of pre- 
requisite principles and concepts (thus causing program error 
rate to increase), the chances are that they will later (on post- 
test) be able to perform adequately on this task either because 
of using knowledge of correct answers supplied in the program 
or because of mental reorganization when they eventually come 
to instruction on the prerequisite skills. 

This distinction between learning sequence and instructional 
sequence is essentially one of means and ends. The sequence of 
frames in a program is a means question; it is concerned with 
how tasks are learned. The necessity for learning sequence as 
defined by Gagné, however, is an ends question; it asks, What 
are the subordinate tasks that must be acquired by the learner 
in order for him to perform this particular terminal task? And, 
as Gagné has pointed out many times (Gagné & Brown, 1961), 
“What is learned is more important than how it is learned 
[p. 320].” It is far more important to be concerned with what 
should be learned, with the identification of all relevant subor- 
dinate behaviors (task analysis) for a particular learning objec- 
tive, than it is to be overly concerned with how (in what order) 
instruction relevant to these subtasks is presented. Note that in 
the typical scramble study all Ss respond, in different sequences, 
to the same set of frames. The what is held constant and the 
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how is manipulated. As has been seen, the usual result of this 
is very little differences in posttest performance. The writer 
would contend that a programmer manipulating frames in search 
of an optimal sequence is wasting his time if those frames do 
not contain instruction pertaining to all subordinate tasks that 
the learner must be able to perform in order to acquire the 
final task. 

Rather than relying on the manipulation of frame sequences 
in order to improve a program, many are suggesting that much 
more work be done prior to even writing the frames. David 
Markle’s description of the development of a first-aid course 
for Bell Telephone (1967) provides an excellent example of this. 
After determining the behaviors desired for the course, test 
items were written for these objectives and tried out on many. 
learners in order to (a) delete skills that could already be con- 
sidered common knowledge to the target population and (b) to 
identify those items which were difficult or ambiguous so as to 
modify the corresponding tasks to be taught. Commenting on 
this preinstruction tryout technique, Markle states, “Ambiguities 
and inefficiencies were revealed in a surprising number of the 
items. Identification of such problems has critical implications 
for the later stages of instructional materials design, because 
an undetected inefficient question will lead to the design of 
concommitantly inefficient instruction [p. 13].’’ Thus an “empi- 
rical specification of objectives” was obtained before any instruc- 
tional materials were developed and tried out. 

In summary it would seem that worthwhile results would come 
from more emphasis on determining and verifying what has to 
be taught in a program rather than focusing primarily on how 
it is taught. Gagné’s concept of task analysis, along with his hier- 
archical taxonomy of learning types (1965) to guide this anal- 
ysis, may be useful. Once the programmer has determined 
precisely what a given group of learners has to be able to do in 
order to perform the terminal tasks, and after he has written 
instructional frames for all of these tasks, it may or may not 
be that relevant how he sequences the instruction. Please note 
that the writer is not advising programmers to ignore frame 
sequence and simply order them at random! Rather, it is being 
questioned whether or not traditional emphasis on frame sequence 
in programed instruction is really as critical to learning as has 
been theorized; and an alternative focus has been suggested. 
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